sumvip .live

Product description



　　Wow. I was literally on HF just now looking for your benchmarks. Heard you did some Behemoth testing recently.

　　","updatedAt":"2024-11-28T00:02:59.294Z","author":{ "_id":"65f2fd1c25b848bd061b5c2e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg","fullname":"Drummer","name":"TheDrummer","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1666}},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9874557852745056},"editors":["TheDrummer"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg"],"reactions":[],"isReport":false}},{ "id":"6747b5df36f67e3c74f80553","author":{ "_id":"65f2fd1c25b848bd061b5c2e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg","fullname":"Drummer","name":"TheDrummer","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1666,"isOwner":true,"isOrgMember":false},"createdAt":"2024-11-28T00:14:23.000Z","type":"comment","data":},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9648801684379578},"editors":["TheDrummer"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg"],"reactions":[],"isReport":false}},{ "id":"6747c3eff6ea2ab0d24bae1d","author":{ "_id":"65644e982bdaccfcd536aff1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4VOzArmrRaX_DUTxGmm59.jpeg","fullname":"Charles McSneed","name":"ChuckMcSneed","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":60,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-28T01:14:23.000Z","type":"comment","data":" data-target="UserMention" class="SVELTE_PARTIAL_HYDRATER contents">

　　to test it) and #3, #1 still remains. Maybe people should loosen minP even more?

　　How was your experience with them during your testing?

　　They were okay during benchmarking, but they really hated different formats, 2407-based ones had no issues with Alpaca, these ones hate it. They wrote inferior poems, just like 2411, very repetitive. Didn't give them proper real-life testing yet to give definitive judgement. Whatever Mistral did, it did not improve the model in all directions.

　　","updatedAt":"2024-11-28T01:14:23.276Z","author":{ "_id":"65644e982bdaccfcd536aff1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4VOzArmrRaX_DUTxGmm59.jpeg","fullname":"Charles McSneed","name":"ChuckMcSneed","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":60}},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9156650304794312},"editors":["ChuckMcSneed"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4VOzArmrRaX_DUTxGmm59.jpeg"],"reactions":[],"isReport":false}},{ "id":"6747fbb4561b1e6f9559b156","author":{ "_id":"65644e982bdaccfcd536aff1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4VOzArmrRaX_DUTxGmm59.jpeg","fullname":"Charles McSneed","name":"ChuckMcSneed","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":60,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-28T05:12:20.000Z","type":"comment","data":{ "edited":false,"hidden":false,"latest":{ "raw":">How was your experience with them during your testing?

　　Less aggressive, makes more spatial mistakes. Might be just confirmation bias though.","html":"

　　How was your experience with them during your testing?

　　Less aggressive, makes more spatial mistakes. Might be just confirmation bias though.

　　","updatedAt":"2024-11-28T05:12:20.776Z","author":{ "_id":"65644e982bdaccfcd536aff1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4VOzArmrRaX_DUTxGmm59.jpeg","fullname":"Charles McSneed","name":"ChuckMcSneed","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":60}},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9882827997207642},"editors":["ChuckMcSneed"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4VOzArmrRaX_DUTxGmm59.jpeg"],"reactions":[],"isReport":false}},{ "id":"67485e178985396b7bd8cbbc","author":{ "_id":"65f2fd1c25b848bd061b5c2e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg","fullname":"Drummer","name":"TheDrummer","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1666,"isOwner":true,"isOrgMember":false},"createdAt":"2024-11-28T12:12:07.000Z","type":"comment","data":{ "edited":false,"hidden":false,"latest":{ "raw":"Interesting stuff, especially the overconfidence metric.

　　Honestly, even my experience with v2.x doesn't feel as solid as v1.x. It's a shame, Mistral probably half-assed their Largestral update. I hope they don't continue to disappoint.

　　Thanks @ChuckMcSneed I'm honored to be part of your experiments and benchmarks 🤗","html":"

　　Interesting stuff, especially the overconfidence metric.

　　Honestly, even my experience with v2.x doesn't feel as solid as v1.x. It's a shame, Mistral probably half-assed their Largestral update. I hope they don't continue to disappoint.

　　Thanks I'm honored to be part of your experiments and benchmarks 🤗

　　","updatedAt":"2024-11-28T12:12:07.003Z","author":{ "_id":"65f2fd1c25b848bd061b5c2e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg","fullname":"Drummer","name":"TheDrummer","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1666}},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9638277888298035},"editors":["TheDrummer"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg"],"reactions":[],"isReport":false}},{ "id":"674860afb4da67d3e0d90145","author":{ "_id":"66826b6dc80934838ff31f2f","avatarUrl":"/avatars/dd5abe99873430067e488dd125c8ca08.svg","fullname":"Niculae","name":"GhostGate","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":9,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-28T12:23:11.000Z","type":"comment","data":},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9846118092536926},"editors":["GhostGate"],"editorAvatarUrls":["/avatars/dd5abe99873430067e488dd125c8ca08.svg"],"reactions":[],"isReport":false}},{ "id":"674863339ff90d8bc8239472","author":{ "_id":"65f2fd1c25b848bd061b5c2e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg","fullname":"Drummer","name":"TheDrummer","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1666,"isOwner":true,"isOrgMember":false},"createdAt":"2024-11-28T12:33:55.000Z","type":"comment","data":},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9829485416412354},"editors":["TheDrummer"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg"],"reactions":[],"isReport":false}},{ "id":"67486a16527ddbbb9e681890","author":{ "_id":"66826b6dc80934838ff31f2f","avatarUrl":"/avatars/dd5abe99873430067e488dd125c8ca08.svg","fullname":"Niculae","name":"GhostGate","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":9,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-28T13:03:18.000Z","type":"comment","data":{ "edited":true,"hidden":false,"latest":{ "raw":"> For me, it feels unavoidably sloppy, harder to wrangle, and slightly lost its creative magic. I've also seen it commit small mistakes that would never happen in 2407 when it comes to following instructions.

　　Strange since new Mistral was supposed to be better at instruction following. Or at least that's what I read in their description. I have yet to try it though. I did try v1 (Q8) for a bit and found it really good at understanding emotions and world immersion (I create my own original worlds and characters since I detest roleplaying in established literature and other forms of media). ","html":"

　　For me, it feels unavoidably sloppy, harder to wrangle, and slightly lost its creative magic. I've also seen it commit small mistakes that would never happen in 2407 when it comes to following instructions.

　　Strange since new Mistral was supposed to be better at instruction following. Or at least that's what I read in their description. I have yet to try it though. I did try v1 (Q8) for a bit and found it really good at understanding emotions and world immersion (I create my own original worlds and characters since I detest roleplaying in established literature and other forms of media).

　　","updatedAt":"2024-11-28T13:03:39.769Z","author":{ "_id":"66826b6dc80934838ff31f2f","avatarUrl":"/avatars/dd5abe99873430067e488dd125c8ca08.svg","fullname":"Niculae","name":"GhostGate","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":9}},"numEdits":1,"identifiedLanguage":{ "language":"en","probability":0.9835860729217529},"editors":["GhostGate"],"editorAvatarUrls":["/avatars/dd5abe99873430067e488dd125c8ca08.svg"],"reactions":[],"isReport":false}},{ "id":"674887833fe49d9343b04b64","author":{ "_id":"660432d2d2e59abb3fd40b8c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/660432d2d2e59abb3fd40b8c/TrvNnR8wHDh9lPHm81JfQ.png","fullname":"David Meriwether","name":"BigHuggyD","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":31,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-28T15:08:51.000Z","type":"comment","data":},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9855437874794006},"editors":["BigHuggyD"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/660432d2d2e59abb3fd40b8c/TrvNnR8wHDh9lPHm81JfQ.png"],"reactions":[],"isReport":false}},{ "id":"67488dd29ff90d8bc8319b81","author":{ "_id":"66826b6dc80934838ff31f2f","avatarUrl":"/avatars/dd5abe99873430067e488dd125c8ca08.svg","fullname":"Niculae","name":"GhostGate","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":9,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-28T15:35:46.000Z","type":"comment","data":},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.986298680305481},"editors":["GhostGate"],"editorAvatarUrls":["/avatars/dd5abe99873430067e488dd125c8ca08.svg"],"reactions":[],"isReport":false}},{ "id":"67498c3d20017d348e8e490c","author":{ "_id":"65f2fd1c25b848bd061b5c2e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg","fullname":"Drummer","name":"TheDrummer","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1666,"isOwner":true,"isOrgMember":false},"createdAt":"2024-11-29T09:41:17.000Z","type":"comment","data":{ "edited":false,"hidden":false,"latest":{ "raw":"@BigHuggyD

　　> I had to accept this was not the character I made

　　Character development is a thing, bro.","html":"

　　I had to accept this was not the character I made

　　Character development is a thing, bro.

　　","updatedAt":"2024-11-29T09:41:17.924Z","author":{ "_id":"65f2fd1c25b848bd061b5c2e","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg","fullname":"Drummer","name":"TheDrummer","type":"user","isPro":true,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":1666}},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9949460029602051},"editors":["TheDrummer"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/65f2fd1c25b848bd061b5c2e/LI9oyVM4OM7jP9dlLeiu_.jpeg"],"reactions":[],"isReport":false}},{ "id":"674995b1f41f1266dfb09399","author":{ "_id":"660432d2d2e59abb3fd40b8c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/660432d2d2e59abb3fd40b8c/TrvNnR8wHDh9lPHm81JfQ.png","fullname":"David Meriwether","name":"BigHuggyD","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":31,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-29T10:21:37.000Z","type":"comment","data":" data-target="UserMention" class="SVELTE_PARTIAL_HYDRATER contents">

　　I had to accept this was not the character I made

　　Character development is a thing, bro.

　　True, true, but not right out of the gate. In this specific example, the assistant is told that they are suspicious of user and user needs to earn their trust. With Monstral, for example, it took 50 messages to convince assistant to trust user. With Behemoth 2.2, they were adopting puppies and picking out furniture for their new life together in 5 messages (exaggeration, but you get my point). I suspect if I were to try 1.1 or 2.1 I would have an experience closer to Monstral.

　　The 2.2 model is still a triumph in its own right, I think. It's style is quite unique, it just seems to suffer some tradeoffs.

　　","updatedAt":"2024-11-29T10:21:37.734Z","author":{ "_id":"660432d2d2e59abb3fd40b8c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/660432d2d2e59abb3fd40b8c/TrvNnR8wHDh9lPHm81JfQ.png","fullname":"David Meriwether","name":"BigHuggyD","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":31}},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9858120679855347},"editors":["BigHuggyD"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/660432d2d2e59abb3fd40b8c/TrvNnR8wHDh9lPHm81JfQ.png"],"reactions":[{ "reaction":"🚀","users":["TheDrummer"],"count":1}],"isReport":false}},{ "id":"6749dacb3399578a6f884a41","author":{ "_id":"6635219caabb42ec31ada8ef","avatarUrl":"/avatars/cc23875e9c4fea8d2e23adbb4d719b42.svg","fullname":"Lightning Missile","name":"lightning-missile","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-29T15:16:27.000Z","type":"comment","data":},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9680359363555908},"editors":["lightning-missile"],"editorAvatarUrls":["/avatars/cc23875e9c4fea8d2e23adbb4d719b42.svg"],"reactions":[],"isReport":false}},{ "id":"674b02a7a05586919e16eb90","author":{ "_id":"660432d2d2e59abb3fd40b8c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/660432d2d2e59abb3fd40b8c/TrvNnR8wHDh9lPHm81JfQ.png","fullname":"David Meriwether","name":"BigHuggyD","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":31,"isOwner":false,"isOrgMember":false},"createdAt":"2024-11-30T12:18:47.000Z","type":"comment","data":},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9669935703277588},"editors":["BigHuggyD"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/660432d2d2e59abb3fd40b8c/TrvNnR8wHDh9lPHm81JfQ.png"],"reactions":[],"isReport":false}},{ "id":"674c06973c4716c22c744f57","author":{ "_id":"65644e982bdaccfcd536aff1","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4VOzArmrRaX_DUTxGmm59.jpeg","fullname":"Charles McSneed","name":"ChuckMcSneed","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":60,"isOwner":false,"isOrgMember":false},"createdAt":"2024-12-01T06:47:51.000Z","type":"comment","data":},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.917899489402771},"editors":["ChuckMcSneed"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/4VOzArmrRaX_DUTxGmm59.jpeg"],"reactions":[],"isReport":false}},{ "id":"674c08ed3965f56df2c049c1","author":{ "_id":"65e1766d7754e5da55f398e6","avatarUrl":"/avatars/9325901eee545ea89cbd3a24a24e57cf.svg","fullname":"DontPlanToEnd","name":"DontPlanToEnd","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":83,"isOwner":false,"isOrgMember":false},"createdAt":"2024-12-01T06:57:49.000Z","type":"comment","data":{ "edited":false,"hidden":false,"latest":{ "raw":"> ...and 2.2 is the worst in writing.

　　2.2 did pretty well in Writing Style, but it performed poorly in the Writing column because it was less willing to write certain things.","html":"

　　...and 2.2 is the worst in writing.

　　2.2 did pretty well in Writing Style, but it performed poorly in the Writing column because it was less willing to write certain things.

　　","updatedAt":"2024-12-01T06:57:49.285Z","author":{ "_id":"65e1766d7754e5da55f398e6","avatarUrl":"/avatars/9325901eee545ea89cbd3a24a24e57cf.svg","fullname":"DontPlanToEnd","name":"DontPlanToEnd","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":83}},"numEdits":0,"identifiedLanguage":{ "language":"en","probability":0.9862331748008728},"editors":["DontPlanToEnd"],"editorAvatarUrls":["/avatars/9325901eee545ea89cbd3a24a24e57cf.svg"],"reactions":[],"isReport":false}},{ "id":"674c818f8472d421eeda25a9","author":{ "_id":"660432d2d2e59abb3fd40b8c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/660432d2d2e59abb3fd40b8c/TrvNnR8wHDh9lPHm81JfQ.png","fullname":"David Meriwether","name":"BigHuggyD","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":31,"isOwner":false,"isOrgMember":false},"createdAt":"2024-12-01T15:32:31.000Z","type":"comment","data":{ "edited":true,"hidden":false,"latest":{ "raw":"Drumma dropped 0.2 with a 2407 base! Time to see how that stew tastes.

　　https://huggingface.co/TheDrummer/Behemoth-123B-v1.2","html":"

　　Drumma dropped 0.2 with a 2407 base! Time to see how that stew tastes.

　　https://huggingface.co/TheDrummer/Behemoth-123B-v1.2

　　","updatedAt":"2024-12-01T15:33:11.469Z","author":{ "_id":"660432d2d2e59abb3fd40b8c","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/660432d2d2e59abb3fd40b8c/TrvNnR8wHDh9lPHm81JfQ.png","fullname":"David Meriwether","name":"BigHuggyD","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":31}},"numEdits":1,"identifiedLanguage":{ "language":"en","probability":0.8744986057281494},"editors":["BigHuggyD"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/660432d2d2e59abb3fd40b8c/TrvNnR8wHDh9lPHm81JfQ.png"],"reactions":[],"isReport":false}}],"pinned":false,"locked":false,"collection":"discussions","isPullRequest":false,"isReport":false},"repo":{ "name":"TheDrummer/Behemoth-123B-v2.2","type":"model"},"activeTab":"discussion","discussionRole":0,"watched":false,"muted":false}">